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ON  THE  APPLICABILITY  OF  BINARY  CORRECTING  CODES  IN 
TRANSMISSION  CHANNELS  QP  DISCRETE  INFORMATION* 

L.  M.  Pink 


Using  asymptotic  expressions  for  the  equivalent 
probability  of  error,  the  conditions  of  ''^accept¬ 
ability”  are  derived  for  a  correcting  code.  These 
conditions  are  satisfied  by  very  few  of  the  codes 
described  in  the  literature.  In  channels  with 
fading,  these  conditions  may  be  made  less  rigid 
if  the  measure  and  decorrelation  of  error  are  taken 
in  constructing  the  code. 


1.  In  order  to  evalxiate  correcting  codes  it  is  convenient  to 
use  the  notion  of  "equivalent  probability  of  error"  [1],  Let  the 
probability  of  incorrect  reception  of  a  symbol  in  a  binary  syiranetrlc 
channel  equal  £,  Using  a  correcting  code  in  this  channel,  it  is 
possible  to  send  a  sequence  of  n  Symbols  with  a  probability  of  correct 
decoding  Q(  n)  .  The  quantity  of  information  contained  in  this  sequence 
is  k  binary  units,  where  k  ^  n.  The  value 

R  =  lim  (  1) 

*  Reported  in  June,  I96I,  at  the  All-Union  Scientific  Session  of 
the  A,  S.  Popov  Scientific  and  Technical  Socle  by  of  Radio  and 
Electronics. 
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will  be,  as  visual,  called  the  redundance  of  the  code. 

This  same  quantity  of  informaition  could  be  sent  with  the  same 
reliability  without  using  a  correcting  code  in  a  binary  symmetric 
channel  if  the  probability  of  correct  reception  of  the  symbol  were 
equal  to  [Q(n)  ]  With  an  increase  in  n  this  probability  con¬ 

verges  on  a  limit  which  we  shall  call  the  equivalent  probability  of 
correct  reception  of  a  symbol.  Its  cong^lement  with  respect  to  unity  i 

1 

Pg.  =  1  -  lim  [Q  (rt)]'/*  =  1  -  Jim  [Q  (n)]  “  (  2) 

rt — n — 

we  shall  call  the  equivalent  probability  of  error. 

For  group  (  systematic)  codes  and  also  for  any  correcting  codes 
in  which  closed  code  combinations  of  a  definite  length  are  used,  one 
may  dispense  with  the  limit  transformation  in  expressions,  having 
taken  as  the  valu  n  the  nvimber  of  symbols  in  the  code  combination. 

In  this,  if  p  «  1,  the  equivalent  probability  of  error  for  all  prac-^ 
tlcal  purposes  coincides  with  the  "specific  probability  of  error," 
Introduced  by  V.  I,  Slforov  [2], 

The  correcting  properties  of  a  code  are  based  on  the  fact  that 
at  least  in  some  range  of  values  of  £  the  Inequality  holds 

Pe  <  P>  (5) 

wherein  the  ratio  — usually  decreases  with  an  Increase  in  p. 

P 

However,  as  has  already  been  repeatedly  mentioned.  Inequality  (5)  is 
not  always  sufficient  to  Justify  the  use  of  a  correcting  code  in  a 
given  channel.  In  most  actual  channels  the  magnitude  of  £  is  a 
function  of  the  dviratlon  of  a  symbol  t.  Therefore,  the  magnitude  of 
£  should  be  made  conpairable  not  with  the  probability  of  incorrect 
reception  of  a  symbol  of  a  correcting  code  p  =  p(  t)  ,  but  with  that 
probability  of  error  p'  which  would  hold  if  binary  symbols  were 
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transmitted  in  an  actixal  channel  at  the  same  rate  at  which  Informatics 
Is  transmitted  xislng  a  correcting  code. 

Obvlo\isly, 

p. ,  p(,i)_p(-^).  (It) 

Only  when  the  Ineqxiallty 

Pe  <  P*  (5) 

Is  fulfilled.  If  orliy  for  all  p*  <  po,  where  po  Is  a  given  number, 
may  one  speak  about  the  advantage  of  using  a  correcting  code.  In  the 
opposite  case.  It  would  be  safer  to  send  Information  at  the  same  rate! 
using  a  code  without  redundancy. 

Let  us  consider  these  codes  satisfying  Condition  ( 5)  at  values 
of  p*  less  than  some  po  as  applicable  for  a  given  code.  The  problem 
of  the  practical  advantage  of  vising  a  code  Is  a  function  also  of 
many  other  conditions.  In  particular  of  the  complexity  and  reliability 
of  the  coding  and  decoding  apparatus,  on  the  value  po>  of  what  degree 
Inequality  (5)  Is  fulfilled,  etc, 

2,  For  Hemming 's  codes*  which  correct  m  errors  In  a  combination 
of  n  symbols,  Q(  n)  Is  expressed  sln^ily: 

Q  (n)  =  V  Ci  p'  (1  -  pT"‘  =  1  -  Ci  p'  (1  -  p)"-'.  .  g. 

If  p  «  1,  In  the  latter  summation  only  the  first  term  may  be 
taken  Into  accovint  and  the  eqvilvalent  probability  of  error  may  be 
expressed  asyn?)totlcally  In  the  form 

*  By  ye'ramlng's  codes  are  meant  systematic  codes  In  which  all 
errors  of  multiple  m  are  corrected  and  errors  of  a  higher  multiple 
are  not  corrected.  "" 
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(7) 


®  K 

Analogously,  for  group  codes[3] 

(3(rt)  =  2a,p'(l -p)"-',  (6‘) 

I’-O 

where  is  the  quantity  of  1-th  correctable  errors.  At  sufficiently 
low  values  of  £, 


P 


e 


K 


{C^‘ 


“m+l 


)P' 


m+1 


(7') 


where  m  Is  the  highest  multiple  of  fully  correctable  errors  (l.e,,  at 
1  ^  m,  =  C^)  .  Apparently,  for  any  correcting  code  these  constants 
a  and  m  may  be  determined  so. 


lim 

p-*0 


ap'"+‘ 


(8) 


3.  Let  the  channel  In  question  be  stationary  and  only  additive 
Interference  In  the  form  of  normal  white  noise  act  In  It.  Then,  as 
Is  known,  \islng  the  optimal  element  coherent  reception,  one  may 
ensure  the  minimal  probability  of  error 

P  =  [1  —  *  ( h)  ] ,  (9) 

where  h  Is  a  value  proportional  to  the  sq\iare  root  of  the  energy  of 
the  signal  element  used  for  transmission  of  one  symbol  ( the  accxjrate 
expression  for  h  Is  a  function  of  the  transmission  method  and  Is  of 
no  further  Interest) , 

0 

Using  the  asyn?)totlc  expression  for  *,  we  obtain  the  dependence 


-4- 


p 


llm 


Viizk 


(10) 


A  similar  dependence  may  be  lyrltten  for  p’,  taking  Expression 
(4)  Into  account.  Since  the  energy  of  the  signal  element  is  propor¬ 
tional  to  T,  then,  preserving  the  symbol  h  when  a  correcting  code  is 

1 

used,  we  find  that  at  an  Increased  t  by  a  factor  of  t=—  "the 

1  -  R 

corresponding  parameter  takes  the  value 


Hence 


h‘ 


h 


lim  — - - 5 - =  1. 

A-**  — /?  [•  h*  ' 

2(1-/?). 


(11) 


(12) 


On  the  other  hand,  from  (8)  and  (10) 


lim 


a 

r  (m+l)A»1 

[  2  J 

=  1. 


(15) 


Comparing  ( 12)  and  ( 1J>)  ,  we  have 


lim  —  =  lim  - . 'nr—TmT^ 

*-*•  p'  y  \—R  (,V2v)"' h” 


If 


R  )  2  y  (14) 

>  m  4-  1,  the  right-hand  member  of  ( 14)  will  Increase 


1  -  R 

without  limit  and,  therefore.  Condition  (5)  will  not  be  fulfilled. 

1 

If,  conversely,  — —  <  m  +  1,  the  limit  of  the  right-hand  member 
1  -  R 

will  equal  zero;  therefore,  at  sufficiently  high  values  of  h 
Inequality  (5)  will  be  fxxlfilled. 


Thus,  for  a  stationary  channel  with  additive  Interference  in  the 
form  of  yrtilte  noise,  the  condition  of  applicability  of  a  correcting 
code  is 


(15) 
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For  groxQ)  codes  this  condition  may  be  written  In  the  form 

(15') 

4.  It  Is  interesting  to  note  that  vrtien  Incoherent  reception  Is 
vised  in  a  stationary  channel  with  additive  normal  noise,  the  condition 
of  applicability  of  a  correcting  code  ( 15)  Is  almost  unchanged. 

Let  us  limit  ourselves  to  the  case  when  an  orthogonal  system  Is 
used  with  an  active  pause  [4].  The  probability  of  error  In  this 
equals 

p  =  ^  exp  (  -  ( 16) 


Taking  (11)  and  (8)  Into  accovint,  we  find 

1 

p'  =  —  exp  r - — — .]  =  i-  (2p)'~'*  , 


lim 


JSl 


2^ 


a  I  m  + 1 


=  1 


and,  therefore. 


(17) 

(18) 


Urn  ^e 


(19) 


Hence,  It  Is  apparent  that  condition  of  applicability  of  a 
correcting  code  Is 


m  > 


1  -  R 


(20) 


or  for  group  codes. 


m>-^ - 

Although  the  equation  m  =  ^  ■  j  In  ( 15)  formally  satisfies  the 

requirement  of  applicability.  In  most  cases  the  equation  corresponds 
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to  codes  whose  use  Is,  In  practice,  unadvantageous.  In  these  cases 
the  difference  between  Pg  and  p*  Is  usually  found  to  be  very  Insig¬ 
nificant  and  manifests  Itself  only  when  the  magnitude  of  p*  Is  so 
small  that  any  congjllcatlon  In  the  apparatus  so  that  Pg  <  p'  does 
not  give  real  advantages.  Thereofre,  for  stationary  channels  with 
additive  white  noise.  In  coherent  as  well  as  Incoherent  reception, 
only  those  codes  for  which  the  Inequality  m  >  ^  Is  fulfilled  are 

of  practical  Interest. 


5.  At  first  glance  It  can  be  shown  that  the  simplest  way  of 
fulfilling  condition  (20)  Is  to  choose  a  code  with  lowest  possible 
redundancy  vrtilch  will  correct  even  Individual  errors  (m  =  1,  R  <  0.5)  • 
However,  this  solution  Is  often  found  to  be  unsuccessful.  Vftien 
redundancy  Is  low,  a  correcting  code  does  not  correct  errors  If  the 
Initial  probability  of  error  Is  not  very  low.  As  an  example  let  us 
examine  the  optimal  group  codes  In  the  work  of  D.  Sleplan  [5].  The 
data  on  these  Is  given  In  Table  1. 

The  applicable  codes  are  denoted  by  asterisks.  Thus,  of  the  42 
group  codes  examined,  only  el^t  are  applicable. 

In  order  to  evaluate  the  effectiveness  of  an  applicable  correct¬ 
ing  code  the  boundary  probability  of  error  po  and  the  relationship 
between  m  and  R  are  of  essential  value.  Limiting  ourselves  to  the 
case  of  Incoherent  reception,  from  ( 16) ,  ( I7) ,  and  ( I9)  we  can  obtain 
the  asyngjtotlc  expression 


In 


In 


+  ( m  •—  R  —  Rm)  ln(  2p ') , 


(21) 


which  Is  the  equation  of  sin  asya^totlc 

Pe 

ence  between  In  — r  and  In  p*.  Figure 

P' 

four  groups  calculated  by  the  accurate 


straight  line  for  the  depend- 
1  shows  this  dependence  for 
formulas  ( 2)  and  ( I7) ,  In  the 


of  groi9  codes  (5.2),  (11.4)  etc,,  the  first  number  represents  n  and 
the  second  k.  Code  (5.2)  Is  not  applicable,  therefore,  for  It 
Pe  >  P'  always.  The  remaining  three  codes  are  applicable,  however, 
code  (7.4)  and  especially  (11.4)  are  not  suitable  In  practice  owing 
to  the  very  low  value  of  po  and  the  small  difference  between  p.  and 

®  I 

p*.  As  Is  apparent  from  Fig.  1,  In  the  range  of  not  very  high  valued 
of  p',  an  asyn5)totlc  straight  line  Is  a  sufficiently  good  approxi¬ 
mation  of  this  dependence.  Therefore,  the  approximate  value  of  the 
bovmdary  probability  of  error  po  may  be  obtained  from  ( 21) ,  having 
made  p^  =  p«  =  po. 

In  ^ - In— —  ln2.  ^  22^ 


TABLE  1 


m 

/? 

n'  ^ 
K 

«  , 

a 

R 

K 

3 

1 

1 

3 

0,67 

2 

6 

2 

1 

3 

0,67 

2 

4 

2 

0 

1 

0.5  , 

1 

6 

.3 

1 

5.3 

0.5 

1 

S. 

.2 

1 

4 

0,6 

1.5 

6 

4 

0 

0,75 

0,67 

2 

5 

3.. 

0 

0.67 

0.4  . 

0.67 

7 

.  .2 

1 

.1,5 

0.71 

2.5 

7 

3 

I 

2.67 

0,57 

1.33 

10 

4 

1 

1,5 

0.6 

1.5 

7 

4 

1 

5,25 

0.43 

0.75* 

10 

5 

1 

4,8 

0,5 

1 

7 

5 

0 

0,8 

0.29 

0.4 

10 

6 

1 

6.67 

0,4 

0,67* 

8 

2 

2 

14,5 

0.75. 

3 

10 

7 

0 

0,43 

0.3 

0,43 

8 

3 

1 

2,67 

0,685 

1.67 

10 

8 

0 

0,87 

0.2 

0,25 

8 

4 

I 

5,25 

0.5 

1 

11 

2 

3 

52 

0,82 

4.5 

8 

5 

0 

0,2 

0.375 

0.6 

11 

3 

2 

13 

0,73 

2,67 

8 

6 

0 

0  83 

0.25 

0,33 

11 

4 

2 

26 

0,64 

1,75* 

9 

2 

2 

10 

0.78 

3.5 

11 

6 

1 

5,83 

0,45 

0,83* 

9 

3 

I 

1 

0.67 

2 

11 

7 

1 

7,29 

0,36 

0,57* 

9 

4 

1 

2,75 

0.56 

1.25 

11 

8 

0 

0.5 

0,27 

0,375 

9 

5 

1 

6 

0.44 

0.8* 

11 

9 

0 

0,9 

0,18 

0,22 

9 

6 

0 

0.33 

0.33 

0.5 

12 

2 

3 

35 

0,83 

5 

9 

7 

0 

0.85 

0.22 

0.29 

12 

3 

2 

6,67 

0,75 

3 

10 

2 

2 

5 

0.8 

4 

12 

7 

1 

6.71 

0,42 

0,71* 

10 

3 

2 

18.7 

0.7 

2.33 

12 

8 

1 

7.87 

0,33 

0.5  • 

1 

12 

9 

0 

0,56 

0,25 

0,33 

The  second  value  describes  the 


degree  of  svilt ability  of  a  code 
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and  Is  the  slope  of  the  straight  line  ( 21) ,  which  may  be  called  the 
"effectiveness"  of  the  code 


E  =  m  —  R  —  Rm,  ( 25) 

or  for  group  codes 

E  =  (m  +  1)—  1.  (25*) 

For  Inapplicable  codes  E  ^  0,  which  coincides  with  condition 
( 20) .  For  applicable  codas,  the  larger  E,  the  more  rapidly  the  gain 
from  using  the  code  at  decreased  p‘  grows.  Table  2  shows  the  calcula¬ 
ted  valiies  of  E,  and  also  the  approximate  values  of  po,  for  the 
eight  applicable  group  codes. 

TABLE  2 


n 

K 

m  . 

a 

E 

P» 

7 

4 

1 

5,25 

1/7 

«  MO-^ 

9 

S 

1  , 

6 

1/9 

«4.10-* 

10 

6 

1 

6,67 

1/5 

s:  M0~® 

11 

4 

2 

26 

1/11 

«  M0~* 

11 

6 

1 

5,83 

I/ll 

«5-10-* 

11 

7 

1 

7,29 

3/11 

a;6-10“* 

12 

•  7 

1 

6.71 

1/6 

«3-10-^ 

1 

8 

1 

7,87 

1/3- 

«5-I0r-* 

Code  (12.8)  has  the  highest  effectiveness.  Code  (11.7)  has 
almost  the  same  effectiveness  and  the  highest  value  of  po.  Only 
these  two  codes  of  all  those  examined  can  provide  any  noticeable 
advantage  In  a  stationary  channel  with  additive  noise  at  sufficiently 
high  slgnal-to-nolse  ratio,  satisfying  the  condition  p*  <  po.  For 
a  more  substantial  Increase  In  noise  stability  when  using  a  group 


code.  It  Is  necessary,  apparently  to  use  longer  code  combinations  at 
higher  values  of  m 


6.  As  Is  knovmi,  dtie  to  fading  many  radio  channels  are  not 
stationary.  In  most  cases  the  relationships  between  the  duration  of 
a  symbol,  the  length  of  a  code  cosablnation  and  the  fading-correlation 
time  are  such  that  for  the  reception  time  of  one  code  combination  the 
channel  may  be  considered  stationary,  l.e.,  the  value  of  £  for  this 
time  Is  almost  unchanged.  Under  these  conditions,  all  these  con¬ 
clusions  become  correct  during  fading  also.  It  is  only  necessary  to 
take  Into  accovint  the  slow  changes  of  £,  as  a  result  of  vrtilch  some 
code  combinations  are  received  with  a  probability  of  error  which  Is 
substantially  greater  than  Its  mean  value.  This  ln^>oses  even  more 
rigid  requirements  vq)on  the  bovindary  value  po.  If  the  code  Is  theo¬ 
retically  applicable,  but  the  value  of  po  does  not  exceed  by  much 
the  mean  probability  of  error  In  an  actual  channel,  then  a  significant 
nvunber  of  code  combinations  will  be  received  at  p‘  >  po  and  the  use 
of  the  code  will  yield  no  gain,  but  will  decrease  noise  stability. 
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The  matter  is  entirely  different  when  the  measures  of  ** error 
randomization**  are  used  In  constructing  the  code.  This  Is  accong)- 
llshed  by  separating  In  time  the  transmission  of  symbols  which  are 
Incorporated  In  a  single  combination  of  the  correcting  code  [1  and 
5].  When  the  spacing  Is  sufficient,  errors  In  the  reception  of 
various  symbols  may  be  considered  Independent,  As  Is  known  |6]  In 
this  the  probability  of  error  ( In  the  case  of  Rayleigh  fading)  Is 


P  = 


h»  +  2 

Taking  (4)  and  (8)  Into  account,  we  have 


(24) 


1  —  R 


(25) 


Thus,  at  any  m  ^  1  and  at  sufficiently  high  values  of  h,  con¬ 
dition  ( 5)  Is  fulfilled  and  the  code  Is  applicable  for  a  channel 
with  fading  during  error  randomization.  In  particular,  any  group 
code  ( transformed  for  error  randomization)  Is  applicable  If  It  permits 
correction  of  all  Individual  errors  In  the  code  combination; 
therefore,  any  Hemming  code  Is  applicable. 

In  practice,  co^^)lete  error  randomization  Is  difficult  to 
attain.  Therefore,  It  Is  advantageous  to  use  those  codes  which  are 
applicable  without  randomization.  Even  If  randomization  Is  found  to 
be  Incomplete,  the  use  of  the  correcting  code  will  not  lead  to  poorer 
reception. 

The  most  simple  error  randomization  Is  attained  by  using  not 
the  group,  but  the  so-called  Iterated  [7]  or  recurrent  [8]  codes. 

The  question  of  the  applicability  of  Iterated  codes,  and  also  codes 
with  correct  error  "splashes,**  will  be  examined  In  another  article. 

Submitted  November  4,  i960 
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DISTRIBUTIOH  LIST 


DEPARTMENT  OF  DEFENSE 

lb*.  Copies 

MAJOR  AIR  COMMANDS 

Nr.  Copies 

AFSC 

SCFTR 

1 

ARO 

1 

HEADQUARTERS  USAF 

A3TIA 

10 

TD-Bla 

3 

AF0IN-3D2 

1 

RADC  (ray) 

1 

SSD  fSSF} 

1 

ASD  (DCF) 

1 

BSD  (BSY) 

1 

OTHER  AGENCIES 

AFSWC  (SV/Y) 

1 

AFf.«TC  (MTW) 

1 

CIA 

1 

APGC  (PGF) 

1 

NSA 

2 

AID 

2 

OTS 

2 

AEC 

2 

PWS 

1 

POE 

1 
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